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INTEGRATED DATA PACKET NETWORK 
USING A COMMON TIME REFERENCE 

RELATED APPLICATIONS 

This application is a continuation of provisional applica- 
tion Ser. No. 60/088,983 filed Jun. 11, 1998. 

BACKGROUND OF THE INVENTION 

This invention relates generally to a method and apparatus 
for integrating the transfer of two traffic types over a data 
packet communications network. More specifically, this 
invention provides timely forwarding and delivery of data 
packets from sources with constant bit rate (CBR) and 
variable bit rate (VBR) over the network and to their 
destination nodes. Consequently, the end-to-end perfor- 
mance parameters, such as, loss, delay and jitter, have either 
deterministic or probabilistic guarantees. 

The proliferation of high-speed communications links, 
fast processors, and affordable, multimedia-ready personal 
computers brings about the need for wide area networks that 
can carry real time data, like telephony and video. However, 
the end-to-end transport requirements of real-time multime- 
dia applications present a major challenge that cannot be 
solved satisfactorily by current networking technologies. 
Such applications as video teleconferencing, and audio and 
video multicasting generate data at a wide range of bit rates 
and require predictable, stable performance and strict limits 
on loss rates, average delay, and delay variations ("jitter"). 
These characteristics and performance requirements are 
incompatible with the services that current circuit and packet 
switching networks can offer. 

Circuit-switching networks, which are still the main car- 
rier for real-time traffic, are designed for telephony service 
and cannot be easily enhanced to support multiple services 
or carry multimedia traffic. Its synchronous byte switching 
enables circuit-switching networks to transport data streams 
at constant rates with little delay or jitter. However, since 
circuit-switching networks allocate resources exclusively 
for individual connections, they suffer from low utilization 
under bursty traffic. Moreover, it is difficult to dynamically 
allocate circuits of widely different capacities, which makes 
it a challenge to support multimedia traffic. Finally, the 
synchronous byte switching of SONET, which embodies the 
Synchronous Digital Hierarchy (SDH), requires increas- 
ingly more precise clock synchronization as the lines speed 
increases [Ballart et al., "SONET: Now It's The Standard 
Optical Network", IEEE Communications Magazine, Vol. 
29 No. 3, March 1989, pages 8-15] [M. Schwartz, "Tele- 
communication Networks: Protocols, Modeling, and 
Analysis", Addison Wesley, Reading Mass., 1987]. 

Packet switching networks like IP (Internet Protocol)- 
based Internet and Intranets [see, for example, A. 
Tannebaum, "Computer Networks" (3rd Ed) Prentice Hall, 
1996] and ATM (Asynchronous Transfer Mode) [see, for 
example, Handel et al., "ATM Networks: Concepts, 
Protocols, and Applications", (2nd Ed.) Addison-Wesley, 
1994] handle bursty data more efficiently than circuit 
switching, due to their statistical multiplexing of the packet 
streams. However, current packet switches and routers oper- 
ate asynchronously and provide best effort service only, in 
which end-to-end delay and jitter are neither guaranteed nor 
bounded. Furthermore, statistical variations of traffic inten- 
sity often lead to congestion that results in excessive delays 
and loss of packets, thereby significantly reducing the fidel- 
ity of real-time streams at their points of reception. In fact, 
under best effort service, the performance characteristics of 
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a given connection are not even predictable at the time of 
connection establishment. 

Efforts to define advanced services for both IP and ATM 
have been conducted in two levels: (1) definition of service, 
and (2) specification of methods for providing different 
services to different packet streams. The former defines 
interfaces, data formats, and performance objectives. The 
latter specifies procedures for processing packets by hosts 
and switches/routers. The types of services that defined for 
ATM include constant bit rate (CBR), variable bit rate 
(VBR) and available bit rate (ABR). For IP, the defined 
services include guaranteed performance (bit rate, delay), 
controlled flow, and best effort [J, Wroclawski, "Specifica- 
tion of the Controlled-Load Network Element Service", 
IETF RFC 2211, September 1997] [Shenker et. al., "Speci- 
fication of Guaranteed Quality of Service", IETF RFC 2212. 
September 1997]. Signaling protocols, e.g., RSVP and 
UNI3.1, which carry control information to facilitate the 
establishment of the desired services, are specified for IP and 
ATM, respectively [R. Braden, "Resource ReSerVation Pro- 
tocol (RSVP) — Version 1 Functional Specification, IETF 
Requestfor Comment RFC2205" September 1997] [Handel 
et al., "ATM Networks: Concepts, Protocols, and 
Applications", (2nd Ed.) Addison-Wesley, 1994]. These 
protocols address the transport of data to one destination 
known as unicast or multiple destinations multicast [S, 
Deering, "Multicast Routing Id Datagram Internet", Ph.D. 
Thesis, Stanford University, December 1991]. In addition, 
SIP, a higher level protocol for facilitating the establishment 
of sessions that use the underlying services, is currently 
under definition under IETF auspices [Handley et al., "SIP- 
Session Initiation Protocol" <draft-draft-ietf-mmusic-sip- 
04.ps>, November 1997]. 

The methods for providing different services under packet 
switching fall under the general title of Quality of Service 
(QoS). Prior art in QoS can be divided into two parts: (1) 
traffic shaping with local timing without deadline 
scheduling, for example [M. G. H. Katevenis, "Fast Switch- 
ing And Fair Control Of Congested Flow In Broadband 
Networks", IEEE Journal on Selected Areas in 
Communications, SAC-5(8): 1315-1326, October 1987; 
Demers et al., "Analysis and Simulation Of A Fair Queuing 
Algorithm", ACM Computer Communication Review 
(SIGCOMM'89), pages 3-12, 1989; S. J. Golestani, 
"Congestion-Free Communication In High-Speed Packet 
Networks", IEEE Transcripts on Communications, COM- 
39(12):1802-1812, December 1991; Parekh et al., "A Gen- 
eralized Processor Sharing Approach To Flow Control — The 
Multiple Node Case", IEEE/ACM T. on Networking, 2(2) 
:137-150, 1994], and (2) traffic shaping with deadline 
scheduling, for example [Ferrari et al., "A Scheme For 
Real-Time Channel Establishment In Wide-Area 
Networks", IEEE Journal on Selected Areas in 
Communication, SAC-8(4):368-379, April 1990; Kandlur et 
al., "Real Tune Communication In Multi-Hop Networks", 
IEEE Trans, on Parallel and Distributed Systems, Vol. 5, No. 
10, pp. 1044-1056, 1994]. Both of these approaches rely on 
manipulation of local queues by each router with little 
coordination with other routers. The Weighted Fair Queuing 
(WFQ), which typifies these approaches, is based on cyclical 
servicing of the output port queues where the service level 
of a specific class of packets is determined by the amount of 
time its queue is served each cycle [Demers et al., "Analysis 
and Simulation Of A Fair Queuing Algorithm", ACM Com- 
puter Communication Review (SIGCOMM'89), pages 
3-12, 1989]. These approaches have inherent limitations 
when used to transport real-time streams. When traffic 
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shaping without deadline scheduling is configured to operate IEEE INFO COM' 96, 1996] provide an abstract (high level) 

at high utilization with no loss, the delay and jitter are description of what is called "RISC-like forwarding", in 

inversely proportional to the connection bandwidth, which which a packet is forwarded, with little if any details, one 

means that low rate connections may experience large delay hop every time frame in a manner similar to the execution 

and jitter inside the network. In traffic shaping with deadline 5 0 f instructions in a Reduced Instruction Set Computer 

scheduling the delay and jitter are controlled at the expense (RISC) machinc [person et al., "Computer Architecture: A 

of possible congestion and loss. Quantitative Approach", Morgan Kaufman Publishers, San 

The recognition that the processing of packets by switches Francisco, 1990]. In U.S. Pat. No. 5,455,701, Eng et al. 

and routers constitutes a performance bottleneck resulted in discloses an apparatus for controlling a high-speed optical 

the development of methods for enhancing performance by 10 sw i tch j ng svstem with pipeline controller for switch control, 

simplifying the processing of packets. Multiprotocol Label In v s Pat No 5,4^779 Vemini et al. disclose a switched 

Switching (MPLS) converts the destination address in the network architecture ^ CO mmon time reference. The time 

packet header into a short tag, which defines the routing of re f e rence is used in order to determine the time in which 

the packet inside the network [Callon et al., "A Proposed multiplicity of nodes can transmit simultaneously over one 

Architecture For MPLS" <draft-ietf-mpls-arch-OO.txt> 15 predefined tree to one destination. At every time 

INTERNET DRAFT, August 1997]. instance the multiplicity of nodes are transmitting to differ- 

The real-time transport protocol (RTP) [H. Schultzrinne ent single destination node, 
et. al, RTP: A Transport Protocol for Real-Time 

Applications, IETF Request for Comment RFC1889, Janu- SUMMARY OF THE INVENTION 

ary 1996] is a method for encapsulating time-sensitive data on r™ . , .« iL j r . j 

, 4 J , Ai . j* j * a* lij*^? The invention describes a method for transmitting and 

packets and attaching to the data time related information c 1 , ^ , u . L1 ,.f . 

f M A ii. 1 • forwarding packets from source nodes with variable bit rates 

like time stamps and packet sequence number. RTP is r\mr>\ t « ** u* * 1 tl 

*i «u t « j f i w (VBR) over a packet switching network. The sources can 

currently the accepted method for transporting real time j c j j 1 

J Tn . / , , 1 generate traffic with predefined average and peak rates. 

streams over IP internetworks and packet audio/video tele- ?, , M *„*u Q „^„Jj, • ~a.,k„„a • 1 „ A _ 

hon based on ITU T H 323 Packets enter the network in predefined time interval and are 

„ . * ' , 25 forwarded over each link inside the network within a peri- 

Synchronous methods are found mostly in circuit odic timc interval if tQere is sufficient h when 

switching, as compared to packet swrtchmg that uses mostly arrivin kets exceed predefined average packets 

asynchronous methods However, some packet switching afe rescheduled for a i, ernaU ve time intervals or discarded. 

synchronous methods have been proposed. IsoEthernet or .„ . . it _ « ... 

ifctc cm n« nccc cm iL*„„*~a /,v.\. The switches of the network maintain a common time 

ihbb oUZ.ya Ihhb 8U2.9a Editor. Integrated service (is): 30 r , . , . , 4 . , ... e A , 

IEEE 802.9a "Isochronous Services With CSMA/CD MAC ™ ° b IT , IT "\ 

Service", IEEE Draft, March 1995] combines CSMA/CD < ro J* " ° P , S T^ loba '^T 8 S f ™) ° r * genera ! ed 

/rt?cn om i\ w u • u i * •* u- a nd distributed internally. The time intervals are arranged m 

(IEEE 802.3), which is an asynchronous packet switching, . . «. . 4 , , • j. . 

-*l kt tct^xt j Tnirt t • < * . £ .... simple periodicity and complex penodicity (like seconds 

with N-ISDN and H.320, which is circuit switching, over a • * f i ia 

existing Ethernet infrastructure (lOBase-T). This is a hybrid 35 mimUcs ot a clocK > 

solution with two distinct switching methods: N-ISDN cir- ^ common time reference is used to determme m 

cuit switching and Ethernet packet switching. Hie two advance the periodic time intervals in which packet are 

methods are separated in the time domain by time division forwarded by each switch on their route from source to 

multiplexing (TDM). The IsoEthernet TDM uses fixed alio- destination. The time interval duration can be longer than the 

cation of bandwidth for the two methods— regardless of 40 time duration required for transmitting a packet, in which 

their utilization levels. This approach to resource partition- case exact P° sition of a P acket 1D the time interval * not 

ing results in undesirable side effect like under-utilization of predetermined. A packet that arrives to an input port is 

the circuit switching part while the asynchronous packet switched to an output port based on specific routing infor- 

switching is over loaded but cannot use the idle resources in mation in the P acket '* header (e.g., IPv4 destination address 

the circuit switching part. 45 in the Intemet > VCI/VPI labels in ATM). 

One approach to an optical network that uses synchroni- Packets that are forwarded inside the network over the 

zation was introduced in the synchronous optical hypergraph same route and in the same P eriodic time intervals constitute 

[Y. Ofek, "The Topology, Algorithms And Analysis Of A a virtual P*P e and ma y share the same pipe-ID. Pipe-ID can 

Synchronous Optical Hypergraph Architecture", Ph.D. be either ex P licit > such a ta S or a la °el that is generated 

Dissertation, Electrical Engineering Department, University so inside ^ network, or implicit such as a group of IP 

of Illinois at Urbana, Report No. UIUCDCS-R-87-1343, addresses. This invention provides deterministic quality of 

May 1987], which also relates to how to integrate packet service guarantees for packets within the predefined average 

telephony using synchronization [Y Ofek, "Integration Of rate * P a <* ets are forwarded free of congestion and 

Voice Communication On A Synchronous Optical with delay jitter smaller than or equal to one time interval 

Hypergraph", INFOCOM J 88, 1988]. In the synchronous 55 Mi other P a ^kets are provided with statistical guarantees, 

optical hypergraph, the forwarding is performed over hyper- This invention facilitates congestion-free forwarding 

edges, which are passive optical stars. In [Li et al, "Pseudo- from one input port to multiple output ports, and 

Isochronous Cell Switching In ATM Networks", IEEE consequently, from one source to multiplicity of destinations 

INFOCOM'94, pages 428-437, 1994; Li et al., "Time- for packets within the predefined average rate. Such packets 

Driven Priority: Flow Control For Real-Time Heteroge- 60 tnat are destined to multiple destinations reach all of their 

neous Internetworking", IEEE INFOCOM'96, 1996] the destinations in predefined rime intervals and with delay jitter 

synchronous optical hypergraph idea was applied to net- that is no larger than one time interval. All other packets are 

works with an arbitrary topology and with point-to-point provided with statistical guarantees, 

finks. The two papers [Li et al., "Pseudo-Isochronous Cell In accordance with the present invention, a method is 

Switching In ATM Networks", IEEE INFO COM 5 94, pages 65 disclosed providing virtual pipes that carry real-time traffic 

428-437, 1994; Li et al., "Time-Driven Priority: Flow over packet switching networks while guaranteeing end-to - 

Control For Real-Time Heterogeneous Internetworking", end performance. The method combines the advantages of 
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both circuit and packet switching. It provides for allocation pages 8-15]. The time resolution of this advanced knowl- 

for the exclusive use of predefined connections and for those edge is much shorter than the data unit transmission time, 

connections it guarantees loss free transport with low delay On the other hand, in accordance with the present invention, 

and jitter. When predefined connections do not use their for each data unit (e.g., a cell) at the time it has been 

allocated resources, other non-reserved data packets can use 5 transmitted from its source, it is possible to know the future 

them without affecting the performance of the predefined time frames that this data unit will be forwarded along its 

connections. On the Internet the non-reserved data packet route. However, the time frame, which constitutes the accu- 

traffic is called "best effort" traffic. In accordance with the racy of this advance timing knowledge, is much larger than 

present invention, the bandwidth allocated to a connection one data unit transmission time. For example, the transmis- 

and the delay and jitter inside the network are independent 10 sion time of an ATM cell (53 bytes) over a gigabit per second 

MPLS can be used by the present invention to identify link is 424 nanoseconds, which is 294 times smaller than a 

virtual pipes. The packet time-stamp that is carried in the typical time frame of 125 microseconds — used in one 

RTP) [H. Schultzrinne et. al, RTP: A Transport Protocol for embodiment of the present invention. There are several 

Real-Time Applications, IETF Request for Comment consequences that further distinguish the present invention 

RFC1889, January 1996] header can be used in accordance 15 from circuit switching: 

with the present invention to facilitate time-based transport. [ n accordance with the present invention, the synchroni- 

Under the aforementioned prior art methods for providing zation requirements are independent of the physical link 

packet switching services, switches and routers operate transmission speed, while in circuit switching the synchro- 

asynchronously. The present invention provides real-time nization becomes more and more difficult as the link speed 

services by synchronous methods that utilize a time refer- 20 increases. 

ence that is common to the switches and end stations In accordance with the present invention, timing infor- 

comprising a wide area network. The common time refer- mation is not used for routing, and therefore, in the Internet, 

ence can be realized by using UTC (Coordinated Universal for example, the routing is done using TP addresses or a 

Time), which is globally available via, for example, GPS tag/label. 

(Global Positioning System— see, for example: http:// X In accordance with the present invention, the Internet 

www.utexas.edu/depts/grg/gcraft/notes/gps/gps.html). By « best effort » packet forwarding strategy can be integrated 

international agreement, UTC is the same all over the world. mt0 tne system. 

UTC is the scientific name for what is commonly called md other aspects and attributes of the prcsent 

GMT (Greenwich Mean Time), the time at the 0 (root) line mven tion will be discussed with reference to the following 

of longitude at Greenwich, England. In 1967, an interna- 30 drawings ^ accompaQying specification, 
tional agreement established the length of a second as the 

duration of 9,192,631,770 oscillations of the cesium atom. BRIEF DESCRIPTION OF THE DRAWINGS 

The adoption of the atomic second led to the coordination of FIG. 1 is a schematic illustration of a virtual pipe and its 

clocks around the world and the establishment of UTC in timing relationship with a common time reference (CTR). 

1972. The Time and Frequency Division of the National 35 Delay is determined by the number of time frames between 

Institute of Standards and Technologies (NIST) (see the forward time out at Node A and the forward time out at 

http:Wwww.boulder.nist.gov/timefreq) is responsible for Node D; 

coordinating with the International Bureau of Weights and pio. 2 is a schematic illustration of multiple virtual pipes; 

Measures (BIPM) in Paris in maintaining UTC. FIG. 3 is a schematic illustration of a switch that uses a 

UTC timing is readily available to individual PCs through 40 common time reference from the GPS (Global Positioning 

GPS cards. For example, TrueTime, Inc/s (Santa Rosa, System) for the timely forwarding of packets disclosed in 

Calif.) PCI-SG provides precise time, with zero latency, to this invention; 

computers that have PCI extension slots. Another way by fig. 4 is a diagram of how time is divided into time 

which UTC can be provided over a network is by using the frames of a predefined duration, and the relationship among 

Network Time Protocol (NTP) [D. Mills, "Network Time me local common time reference (CTR) on the switches, and 

Protocol" (version 3) IETF RFC 1305]. However, the clock how the multiplicity of local times is projected on the 

accuracy of NTP is not adequate for interswitch real-time axis; 

coordination, on which this invention is based. FIG. 5 is a schematic illustration of how the common time 

In accordance with the present invention, the use of 5Q reference is organized into contiguous time -cycles of k 

reserved resources is allowed by all packet traffic whenever time-frames each and contiguous super-cycle of 1 time- 

the reserved resources are not in use. cycles each; 

A key difference between the synchronous optical hyper- FIG. 6 is a schematic illustration of the relationship of the 
graph and the present invention is the forwarding of packets network common time reference and UTC (Coordinated 
over simple point-to-point edges in this invention. The 55 Universal Time), such that, each time-cycle has 100 time- 
pipeline in accordance with the present invention is used for frames, of 125 //sec each, and 80 time-cycles are grouped 
the forwarding of packets inside the network, not for switch into one super-cycle of one second; 
control as in the Eng et al. patent- FIG. 7 is a schematic illustration of a data packet pipeline 
Although the present invention relies on time to control forwarding over a virtual pipe with common time reference 
the flow of packets inside the network in a similar fashion as 60 (CTR); 

in circuit switching, there are major differences between the FIG. 8 is an illustrative example of the mapping of the 

two approaches. In circuit switching, for each data unit (e.g., time frames in and the time frames out of a node on a virtual 

a byte) at the time it has been transmitted from its source, it pipe with variable bit rate. The mapping repeats itself in 

is possible to predict deterministically the future times it will every time cycle and the mapping of the time cycle repeats 

be transmitted from any switch along its route [Ballart et al., 65 itself in every super cycle. All additions are modulo 80 and 

"SONET: Now It's The Standard Optical Network'', IEEE variable bit rate packets forwarding time can be delayed 

Communications Magazine, Vol. 29 No. 3, March 1989, until the next time frame of the same virtual pipe; 
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FIG. 9 is an illustration of a serial transmitter and a serial input port of a switch, is switched to an output port based on 

receiver; specific routing information in the packet's header (e.g., 

FIG. 10 is a table of the 4B/5B encoding scheme for IPv4 destination address in the Internet, VCI/VPI labels in 

data-used by the AM7968TAXI chip set; ^™)- . E «* swit <; h al ° n 8 a route J from a to L a 

. ,. . s destination forwards packets in periodic time intervals that 

RG. 11 is a table of the 4B/5B encoding scheme for are predefined using the common ^ reference. The time 

control signals, such as, the time frame delimiter (TFD) mterval duration can be longer than the time duration 

used by the AM7968; required for transmitting a packet, in which case the exact 

FIG. 12 is a schematic illustration of an input port with a position of a packet in the time interval is not predetermined, 

routing controller; Packets that are forwarded inside the network over the 

FIG. 13 is a schematic diagram of the routing controller. s *me route and in the same periodic time intervals constitute 

This unit determines to which output port an incoming data a virtual pip c and SQare * c same pipc-ID. Pipe-ID can be 

packet should be switched to and attaches the time of arrival either exp^t, such as a tag or a label that is generated inside 

(TOA) information to the data packet header; the network, or implicit such as a group of IP addresses. A 

lit j -| , .« . c * virtual pipe can be used to transport data packets from 

FIGS. 14A and 14B are schematic illustrations of two 15 l4 . , r r , . . . , \. .. \ . . . . 

generic data packet headers with virtual pipe ID (PID) and multl P le ^ces and I to multiple destinations. A virtual pipe 

• - t L'</n\ /a\ 14 *u * *■ . a ij j provides deterministic quality of service guarantees. The 

priority bit (P): (A) a packet without a time-stamp field and f. . 4 , . , . . n - 4 i c , •*> i , ■ 

ro\ , / V , a u rpi ' j • i time interval in which a switch forwards a specific packet is 

(B) a packet with a time -stamp field. This drawing also , . . , , , Tr , . . .« * . tu 

V' I .. / t c i determined by the packet s pipe-ID, the time it reaches the 

shows how the common time-reference value, time of arrival . , , ' r < . r v, ' r 

a™a\ • ** i. j t. *u .i» switch, and the current value of the common tune reference. 

(TOA), is attached by the routing controller; 20 T ' , ... , . r 

v 7 In accordance with the present invention, congestion-free 

FIG. 15 is a table classifying the data packets; p acket switc bing is provided for pipe-IDs in which capacity 

FIG. 16 is a description of the routing controller opera- in their corresponding forwarding links and time intervals is 

tion; reserved in advance. Furthermore, packets that are trans- 

FIG. 17 is a schematic illustration of an output port with 25 ferred over a virtual pipe reach their destination in pre- 

a scheduling controller and a serial transmitter; defined time intervals, which guarantees that the delay jitter 

FIG. 18 is a functional description of the transmit sched- ^ smaller or e q ual to one time interval, 

uler with a double buffer, where each buffer is divided into Packets that are forwarded from one source to multiple 

two parts, one for constant bit rate (CBR) and the other for destinations share the same pipe ID and the links and time 

variable bit rate (VBR); 30 intervals on which they are forwarded comprise a virtual 

FIG. 19 is a description of the double-buffer scheduling ! ree ' ™ s States congestion-free forwarding from one 

and congestion controller 46 operation; m P m P ort t0 multl P le °^put ports, and consequently, from 

i-™ -ia ■ r 11 . r i( A . i one source to multiplicity of destinations. Packets that are 

MG. 20 is a functional description of the transmit buffer A , , u . , r , / 4 . , „ c . , 

..... , . r . ii destined to multiple destinations reach all of their destma- 

and schedulme and congestion controller: . , £ * . . < , ... , « 

& & 7 „ tions in predefined time intervals and with delay jitter that is 

FIG. 21 is a flow diagram describing the packet sched- 35 no k ^ one time interval, 

uling and rescheduling controller operation for computing A m is ided fof m data transfer of daU 

the forward time of a packet based on the following input kets from a SQUrce tQ a destinatioiK ^ transfer of the 

parameters: pipe-ID 35C Time of arrival 35T and the data kcts [s ided durin a cdcfined timc mtcrval 

common time reference 002; comprised of a plurality of predefined time frames. Tne 

FIG. 22 is a flow diagram describing the select buffer and systcm ^ farthcr comprised of a plurality of switches. A 

congestion controller 45D; v i rtua l p i pe j s comprised of at least two of the switches 

FIG. 23 is a description of the real-time protocol (RTP) interconnected via communication links in a path. A corn- 
packet header with time-stamp 35TS field of 32 bits; and mon time reference signal is coupled to each of the switches, 

FIG. 24 is a flow diagram describing the packet sched- 45 and a time assignment controller assigns selected predefined 

uling and rescheduling controller operation for computing time frames for transfer into and out from each of the 

the forwarding time of a packet based on the following input respective switches responsive to the common time refer- 

parameters: pipe-ID, Time of arrival 35T, common time ence signal. For each switch, there is a first predefined time 

reference 002 and a time-stamp 35TS. frame within which a respective data packet is transferred 

50 into the respective switch, and a second predefined time 

DETAILED DESCRIPTION OF THE f rame within which the respective data packet is forwarded 

PREFERRED EMBODIMENTS out 0 f the reS pective switch. The time assignment provides 

While this invention is susceptible of embodiment in consistent fixed intervals between the time between the input 

many different forms, there is shown in the drawing, and will to and output from the virtual pipe, 

be described herein in detail, specific embodiments thereof 55 In a preferred embodiment, there is a predefined subset of 

with the understanding that the present disclosure is to be the predefined time frames during which the data packets are 

considered as an exemplification of the principles of the transferred in the switch, and for each of the respective 

invention and is not intended to limit the invention to the switches, there are a predefined subset of the predefined time 

specific embodiments illustrated. frames during which the data packets are transferred out of 

The present invention relates to a system and method for 60 tne switch, 

transmitting and forwarding packets over a packet switching Each of the switches is comprised of one or a plurality of 

network. The switches of the network maintain a common addressable input and output ports. A routing controller 

time reference, which is obtained either from an external maps each of the data packets that arrives at each one of the 

source (such as GPS — Global Positioning System) or is input ports of the respective switch to a respective one or 

generated and distributed internally. The time intervals are 65 more of the output ports of the respective switch, 

arranged in simple periodicity and complex periodicity (like For each of the data packets, there is an associated time 

seconds and minutes of a clock). A packet that arrives to an of arrival to a respective one of the input ports. The time of 
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arrival is associated with a particular one of the predefined The communication links can be of fiber optic, copper, 

time frames. For each of the mappings by the routing and wireless communication links for example, between a 

controller, there is an associated mapping by a scheduling ground station and a satellite, and between two satellites 

controller, which maps of each of the data packets between orbiting the earth. The communication link between two 

the time of arrival and forwarding time out. The forwarding 5 no des does not have to be a serial communication link. A 

time out is associated with a specified predefined time frame. parallel co mmun ication link can be used-^uch link can 

In the preferred embodiment, there are a plurality of the carrv simultaneously multiple data bits, associated clock 

virtual pipes comprised of at least two of the switches signa]> and associated signals, 

interconnected via communication links in a path. The ™, , . t . , T 4 . , /1TVV - t 

communication link is a connection between two adjacent in ^ data can * e Jl f me ^ p ^l <"? data 

switches; and each of the communications links can be used 30 P^kets and asynchronous transfer mode (ATM) cells, and 

simultaneously by at least two of the virtual pipes. Multiple can 1x5 folded over the same virtual pipe having an 

data packets can be transferred utilizing at least two of the associated pipe identification (PID). The PID can be an 

virtual pipes. Internet protocol (IP) address, Internet protocol group mul- 

There is a fixed time difference, which is constant for all ticast address, an asynchronous transfer mode (ATM), a 

switches, between the time frames for the associated time of 15 virtual circuit identifier (VCI), and a virtual path identifier 

arrival and forwarding time out for each of the data packets. (V? 1 )* or (used in combination as VCl/VPI). 

The fixed time difference is a variable time difference for The routing controller determines two possible associa- 

some of the switches. A predefined interval is comprised of tions of an incoming data packet: (i) the output port, and (ii) 

a fixed number of contiguous time frames comprising a time the time of arrival (TO A). The TO A is then used by the 

cycle. Data packets that are forwarded over a given virtual 20 scheduling controller for determining when a data packet 

pipe are forwarded from an output port within a predefined should be forwarded by the select buffer controller to the 

subset of time frames in each time cycle. Furthermore, the next sw jtch in the virtual pipe. The routing controller utilizes 

number of data packets that can be forwarded in each of the at least one of Internet prot ocol version 4 (IPv4), Internet 

predefined subset of time frames for a given virtual pipe is protocol yersion 6 addresses? Internet protocol group 

also predefined. 25 mu iticast address, Internet MPLS (multi protocol label 

The time frames associated with a particular one of the swapping or tag switching) labels, ATM virtual circuit 

switches within the virtual pipe are associated with the same idcntifier and th idcntificr (VC I/VPI), and IEEE 

switch for all the time cycles and are also associated with 802 MAC (media ^ cQntrol) ^ d ^ { 

one ot input into or output from the particular respective ? • ♦ _* « rr 

switch * r r i wm an inpu t por t to an output port. 

TTiere is a constant fixed time between the input into and 3 ° Each of the data P ackets » comprised of a header, which 

output from a respective one of the switches for each of the includes an associated time stamp. For each of the mappings 

time frames within each of the time cycles. A fixed number b V the routin S controller, there is an associated mapping by 

of contiguous time cycles comprise a super cycle, which is tne scheduling and rescheduling controller, of each of the 

periodic. Data packets that are forwarded over a given 35 data packets between the respective associated time-stamp 

virtual pipe are forwarded from an output port within a and an associated forwarding time out, which is associated 

predefined subset of time frames in each super cycle. with one of the predefined time frames. The time stamp can 

Furthermore, the number of data packets that can be for- record the time in which a packet was created by its 

warded in each of the predefined subset of time frames application. 

within a super cycle for a given virtual pipe is also pre- 4Q In one embodiment, the time-stamp is generated by an 

defined. Internet real-time protocol (RTP), and by a predefined one of 

In the preferred embodiment, the common time reference the switches. The time-stamp can be used by a scheduling 

signal is coupled from a GPS (Global Positioning System), controller in order to determine the forwarding time of a data 

and is in accordance with the UTC (Coordinated Universal packet from an output port. 

Time) standard. The UTC time signal does not have to be 45 Each of the data packets originates from an end station, 

received directly from GPS, such signal can be received by and the time-stamp is generated at the respective end station 

using various means, as long as the delay or time uncertainty for inclusion in the respective originated data packet. Such 

associated with that UTC time signal does not exceed half a generation of a time-stamp can be derived from UTC either 

time frame. by receiving it directly from GPS or by using the Internet's 

In one embodiment, the super cycle duration is equal to 50 Network Time Protocol (NTP). 

one second as measured using the UTC (Coordinated Uni- The following description illustrates a preferred embodi- 

versal Time) standard. The super cycle can also be equal to ment and implementation of the system disclosed in accor- 

multiple UTC seconds or a fraction of a UTC second. dance with the present invention. In FIGS. 1-8, the prin- 

A select buffer controller maps one of the time frames for ciples of operation and implementation are presented and 

output from a first switch to a second time frame for input 55 discussed. In FIGS. 9-24, the details of one embodiment's 

via the communications link to a second switch. The select design are given. 

buffer controller uses the UTC time signal in order to In accordance with the present invention, a system is 

identify the boundaries between two successive time frames. provided for transferring data packets across a data network 

The select buffer controller insert a time frame delimiter while maintaining for reserved data traffic constant bounded 

(TFD) signal into the transmission link in order to the signal 60 jitter (or delay uncertainty) and no congestion-induced loss 

the second switch the exact boundary between two time of data packets. Such properties are essential for many 

frames. multimedia applications, such as, telephony and video tele- 

Each of the data packets is encoded as a stream of data, conferencing, 

and a time frame delimiter is inserted into the stream of data In accordance with the design, method, and illustrated 

responsive to the select buffer controller. This can be imple- 65 implementation of the present invention, one or a plurality 

mented by using a redundant serial codewords as it is later of virtual pipes 25 are provided, as shown in FIG. 1 and FIG. 

explained. 2, over a data network with general topology. Such data 
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network can span the globe. Each virtual pipe 25 is con- 
structed over one or more switches 10, shown in FIG. 1, 
which are interconnected via communication links 41 in a 
path. 

FIG. 1 illustrates a virtual pipe 25 from the output port 40 
of switch A, through switches B and C. This virtual pipe 
ends at the output port 40 of node D. The virtual pipe 25 
transfers data packets from at least one source to at least one 
destination. 

FIG. 2 illustrates three virtual pipes: virtual pipe 1 from 
the output of switch A to the output of switch D, virtual pipe 
2 from the output of switch B to the output of switch D, and 
virtual pipe 3 from the output of switch A to the output of 
switch C. 

The data packet transfers over the virtual pipe 25 via 
switches 10 are designed to occur during a plurality of 
predefined time intervals, wherein each of the predefined 
time intervals is comprised of a plurality of predefined time 
frames. The timely transfers of data packets are achieved by 
coupling a common time reference 002 (CTR) signal to each 
of the switches 10. 

FIG. 3 illustrates the structure of a pipeline switch 10. The 
switch 10 is comprised of one or a plurality of input ports 30, 
one or a plurality of output ports 40, switching fabric 50, and 
global positioning system (GPS) time receiver 20 with a 
GPS antenna 001. The GPS time receiver provides a com- 
mon time reference signal (CTR) 002 to all input and output 
ports. 

The Common Time Reference (CTR) 002 

As shown in FIG. 4, the common time reference 002 that 
is coupled to the switches 10 provides the following prop- 
erty: the local clock ticks 004, shown in FIG. 4, at all the 
pipeline switches (e.g., switches A, B, C, and D in FIGS. 1 
and 2) when projected on the real-time axis 005 will all 
occur within predefined synchronization envelopes 003. In 
other words, the local clock ticks 004 occur within the 
synchronization envelopes 003, and therefore, outside to the 
synchronization envelopes all local clocks have the same 
clock value. 

The common time reference is divided in a predefined 
manner into time frames, Tf, of equal duration, as shown in 
FIG. 4, typically TM25 microseconds. The time frames are 
grouped into time cycles. Each time cycle has predefined 
number of time frames. 

Referring to FIG. 5, there are k time frames in each time 
cycle. Contiguous time cycles are grouped together into 
contiguous super cycles, and as shown in FIG. 5, there are 
1 time cycles in each super cycle. 

FIG. 6 illustrates how the common time reference can be 
aligned with the UTC (Coordinated Universal Time) stan- 
dard. In this illustrated example, every duration of every 
super cycle is exactly one second as measured by UTC. 
Moreover, the beginning of each super cycle coincides with 
the beginning of a UTC second, as shown in FIG. 6. 
Consequently, when leap seconds are inserted or deleted for 
UTC corrections (due to changes in the earth rotation 
period) the cycle and super cycle periodic scheduling will 
not be affected. 

The time frames, time cycles, and super cycles are asso- 
ciated in the same manner with all respective switches 
within the virtual pipe at all times. 
Pipeline Forwarding 

Pipeline forwarding relates to data packets being for- 
warded across a virtual pipe 25 with a predefined delay in 
every stage (either across a communication link 41 or across 
a switch 10 from input port 30 to output port 40). Data 
packets enter a virtual pipe 25 from one or more sources and 
forwarded to one or more destinations. 
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This sort of pipeline forwarding used in accordance with 
the present invention is illustrated in FIG. 7. Data packet 
41A is forwarded out of switch A during time frame t-1. 
This data packet 41A will reach switch B after a delay of 
T-ab, This data packet 41A will be forwarded out of switch 
B as data packet 41B during time frame t+1 and will reach 
switch C after a delay of T-bc. This data packet 41B will be 
forwarded out of switch C as data packet 41 C during time 
frame t+5. Data packet 41C will reach switch D after a delay 
of T-cd. Consequently, the delay from the output of switch 
A to the output of switch C is 6-t+5-(t-l) time frames. As 
illustrated in FIG. 7, all data packets that are forwarded over 
that virtual pipe will have a delay of six time frames from the 
output of switch A to the output of switch C. 

Referring again to FIG. 1, the timely pipeline forwarding 
of data packets over the virtual pipe 25 is illustrated. A data 
packet is received by one of the input ports 30 of switch A 
at timeframe 1, and is forwarded along this virtual pipe 25 
in the following manner: (i) the data packet 41 A is for- 
warded from the output port 40 of switch A at time frame 2 
of time cycle 1, (ii) the data packet 41B is forwarded from 
the output port 40 of switch B, after 18 time frames, at time 
frame 10 of time cycle 2, (iii) the data packet 41C is 
forwarded from the output port 40 of switch C, after 42 time 
frames, at time frame 2 of time cycle 7, and (iv) the data 
packet 41D is forwarded from the output port 40 of switch 
D, after 19 time frames, at time frame 1 of time cycle 9. 

As illustrated in FIG. 1, 

All data packets enter the virtual pipe 25 (i.e., forwarded 
out of the output port 40 of switch A) periodically at the 
second time frame of a time cycle, are output from this 
virtual pipe 25 (i.e., are forwarded out of the output port 
40 of switch D) after 79 time frames. 

The data packets that enter the virtual pipe 25 (i.e., 
forwarded out of the output port 40 of switch A) can 
come from one or more sources and can reach switch 
A over one or more input links 41. 

The data packets that exit the virtual pipe 25 (i.e., for- 
warded out of the output port 40 of switch D) can be 
forwarded over plurality of output links 41 to one of 
plurality of destinations. 

The data packets that exit the virtual pipe 25 (i.e., for- 
warded out of the output port 40 of switch D) can be 
forwarded simultaneously to multiple destinations, 
(i.e., multicast (one-to-many) data packet forwarding). 

The communication link 41 between two adjacent ones of 
the switches 10 can be used simultaneously by at least 
two of the virtual pipes. 

In FIG. 2, where there are three virtual pipes: 

The three virtual pipes can multiplex (i.e., mix their 
traffic) over the same communication links. 

The three virtual pipes can multiplex (i.e., mix their 
traffic) during the same time frames and in an arbitrary 
manner. 

The same time frame can be used by multiple data packets 
from one or more virtual pipes. 
Virtual Pipe Capacity Assignment 

For each virtual pipe there are predefined time frames 
within which respective data packets are transferred into its 
respective switches, and separate predefined time frames 
within which the respective data packets are transferred out 
of its respective switches. Though the time frames of each 
virtual pipe on each of its switches can be assigned in an 
arbitrary manner along the common time reference, it is 
convenient and practical to assign time frames in a periodic 
manner in time cycles and super cycles. 
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FIG. 8 illustrates the timing of a switch of a virtual pipe 
wherein there are a predefined subset of time frames (i, 75, 
and 80) of every time cycle, during which data packets are 
transferred into that switch, and wherein for that virtual pipe 
there are a predefined subset time frames (i+3, 1, and 3) of 5 
every time cycle, during which the data packets are trans- 
ferred out of that switch. If each of the three data packets has 
125 bytes or 1000 bits, and there are 80 time frames of 125 
microseconds in each time cycle (i.e., time cycle duration of 
10/*sec), then the bandwidth allocated to this virtual pipe is 10 
300,000 bits per second. 

In general, the bandwidth or capacity allocated for a 
virtual pipe is computed by dividing the number of bits 
transferred during each of the time cycles by the time cycle 
duration. In the case of a super cycle, the bandwidth alio- 15 
cated to a virtual pipe is computed by dividing the number 
of bits transferred during each of the super cycles by the 
super cycle duration. 

The switch 10 structure, as shown in FIG. 3, can also be 
referred to as a pipeline switch, since it enables a network 20 
comprised of such switches to operate as a large distributed 
pipeline architecture, as it is commonly found inside digital 
systems and computer architectures. 

Each pipeline switch 10 is comprised of a plurality of 
addressable input ports 30 and output ports 40. As illustrated 25 
in FIG. 12, the input port 30 is further comprised of a routing 
controller 35 for mapping each of the data packets that 
arrives at each one of the input ports to a respective one of 
the output ports. As illustrated in FIG. 16, the output port 40 
is further comprised of a scheduling controller and transmit 30 
buffer 45. An output port 40 is connected to an input port 30 
via a communication link 41, as shown in FIG. 9. The 
communication link can be realized using various technolo- 
gies compatible with the present invention. 

As shown in FIG. 3, the common time reference 002 is 35 
provided to the input ports 30 and output ports 40 from the 
GPS time receiver 20, which receives its timing signal from 
the GPS antenna 001. GPS time receivers are available from 
variety of manufacturers, such as, Truellme, Inc. (Santa 
Rosa, Calif.). With such equipment, it is possible to maintain 40 
a local clock with accuracy of ±1 microsecond from the 
UTC (Coordinated Universal Time) standard everywhere 
around the globe. 

The Communication Link and Time Frame Delimiter 
Encoding 45 

The communication links 41 used for the system dis- 
closed is in this invention can be of various types: fiber optic, 
wireless, etc. The wireless links can be between at least one 
of a ground station and a satellite, between two satellites 
orbiting the earth, or between two ground stations, as 50 
examples. 

Referring to FIG. 9, a serial transmitter 49 and serial 
receiver 31 are illustrated as coupled to each link 41. A 
variety of encoding schemes can be used for a serial line link 
41 in the context of this invention, such as, SONET/SDH, 55 
8B/10B Fiber Channel, 4B/5B FDDI. In addition to the 
encoding and decoding of the data transmitted over the serial 
link, the serial transmitter/receiver (49/31) sends/receives 
control words for a variety of control purposes, mostly 
unrelated to the present invention description. However, one 60 
control word, time frame delimiter (TFD), is used in accor- 
dance with the present invention. The TFD marks the 
boundary between two successive time frames and is sent by 
a serial transmitter 49 when a CTR 002 clock tick occurs in 
a way that is described hereafter as part of the output port 65 
operation. It is necessary to distinguish in an unambiguous 
manner between the data words, which carry the 



information, and the control signal or words (e.g., the TFD 
is a control signal) over the serial link 41. There are many 
ways to do this. One way is to use the known 4B/5B 
encoding scheme (used FDDI). In this scheme, every 8 -bit 
character is divided into two 4-bit parts and then each part 
is encoded into a 5 -bit codeword that is transmitted over the 
serial link 41. 

FIG. 10 illustrates an encoding table from 4-bit data to 
5 -bit serial codewords. The 4B/5B is a redundant encoding 
scheme, which means that there are more codewords than 
data words. Consequently, some of the unused or redundant 
serial codewords can be used to convey control information. 

FIG. 11 is a table with 15 possible encoded control 
codewords, which can be used for transferring the time 
frame delimiter (TFD) over the serial link. The TFD transfer 
is completely transparent to the data transfer, and therefore, 
it can be sent in the middle of the data packet transmission 
in a non-destructive manner. 

When the communication links 41 are SONET/SDH, the 
time frame delimiter cannot be embedded as redundant 
serial codewords, since SONET/SDH serial encoding is 
based on scrambling with no redundancy. Consequently, the 
TFD is implemented using the SONET/SDH frame control 
fields: transport overhead (TOH) and path overhead (POH). 
Note that although SONET/SDH uses a 125 microseconds 
frame, it cannot be used directly in accordance with the 
present invention, at the moment, since SONET/SDH 
frames are not globally aligned and are also not aligned to 
UTC. However, if SONET/SDH frames are globally 
aligned, SONET/SDH can be used compatibly with the 
present invention. 
The Input Port 

The input port 30, shown in FIG. 12, has three parts: serial 
receiver 31, routing controller 35 and separate queues to the 
output ports 36. The serial receiver 31 transfers to the 
routing controller 35 the data packets and the time frame 
delimiters. 

The routing controller is constructed of a central process- 
ing unit (CPU), a random access memory (RAM) for storing 
the data packet, read only memory (ROM) for storing the 
routing controller processing program, and routing table is 
used for determining the output port that the incoming data 
packet should be switched to. The incoming data packet 
header includes a virtual pipe identification — PID 35 C in 
FIG. 14, that is used to lookup in the routing table 35D the 
address 35E of the queue the incoming data packet should be 
transferred into 36. Before the packet is transferred into its 
queue 36 the time of arrival (TO A) 35T in FIG. 14 is 
attached to the packet header. The TOA35T will be used by 
the scheduling controller 45 in FIG. 17 in the computation 
of the forwarding time out of the output port. 

The data packet can have various formats, such as, 
Internet protocol version 4 (IPv4), Internet protocol version 
6 (IPv6), asynchronous transfer mode (ATM) cells. The data 
packets PID can be determined by one of the following: an 
Internet protocol (IP) address, an asynchronous transfer 
mode (ATM) a virtual circuit identifier, and a virtual path 
identifier (VCI/VPI), Internet protocol version 6 (IPv6) 
addresses, Internet MPLS (multi protocol label swapping or 
tag switching) labels, and IEEE 802 MAC (media access 
control) address. 

FIG. 15 is a table for defining two bits, PI and P2, in the 
packet headers in FIG. 14. The two bits classify three types 
of data packets: P1/P2 are 00 constant bit rate (CBR) data 
traffic, P1,P2 are 01 variable bit rate (VBR) data traffic, 
P1,P2 are 10 "best effort" data traffic. The above classifica- 
tion is used by the program executed by the routing con- 
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troller 35B, as shown in FIG. 16, in order to determine into 
which of the three parts of the queue to the output port 36, 
shown in FIG. 13, the data packet should be switched into. 

FIG. 16 is a detailed description of the program executed 
by the routing controller 35B. The program is responsive to $ 
two basic events from the serial receiver 31: receive time 
frame delimiter TFD 35-01, and receive data packet 35-02. 
After receiving a TFD the routing controller computes the 
time of arrival (TOA) 35T value 35-03 in FIG. 14, that is 
attached to the incoming data packets. For this computation 
it uses a constant, Dconst, which is the time difference 
between the common time reference (CTR) 002 tick and the 
reception of the TFD at time t2 (note that the TFD was 
generated on an adjacent switch by the CTR 002 on that 
node). This time difference is caused by the fact that the 
delay from the serial transmitter 49 to the serial receiver 31 35 
is not an integer number of time frames. When data packet 
is received 35-02 the routing controller 35B executes three 
operations 35-04 in FIG. 16: attach the TOA, lookup the 
address of the queue 36 using the PID, and storing the data 
packet in the queue 36 to the output port 36, while using 20 
P1/P2 in the header, in FIG. 14, in order to determine in what 
part, CBR/VBR/Best effort, of that queue to store the 
incoming data packet. 
Multicast — One-to-many Forwarding 

Data packets with certain predefined PIDs are forwarded 25 
more that output ports, in which case the table 35D has 
multiple queue values. The data packet is then copied to all 
those queues by the routing controller 35B, as was previ- 
ously described, and consequently the data packet will be 
forwarded to multiple output ports. 30 
The Switching Fabric 

There are various ways to implement a switching fabric, 
however, since the switching fabric is not the subject of this 
invention it will be described only briefly. The main property 
that the switching fabric should ensure is that packets for 35 
which the priority bits P1/P2 are 00 (see FIG. 14), gets high 
priority (i.e., reserved traffic), and will be switched into the 
output port in a constant bounded delay — measured in time 
frames. 

This is possible when the packets in the input ports are 40 
already separated into queues to their respective output 
ports, then by using the Clos theorem in the time domain 
(see J. Y Hui "Switching and Traffic Theory for Integrated 
Broadband Networks," page 65) the delay can be bounded 
by two time frames — one time frame at the input port and 45 
one time frame to get across the switching fabric. Other 
implementation can be based on shared bus with round robin 
service of the high priority data packets or on a crossbar 
switch 

The Output Port 50 

The output is depicted in FIG. 17, it has two parts a 
scheduling controller with a transmit buffer 45, and a serial 
transmitter 49, which was described before. The scheduling 
controller performs a mapping of each of the data packets 
between the associated respective time of arrival (TOA) and 55 
an associated forwarding time out of the output port via the 
serial transmitter. The forwarding lime is determined rela- 
tive to the common time reference (CTR) 002, as it is 
described in the following. 

Three output port configurations are disclosed: double- 60 
buffer scheduling and congestion controller, as depicted in 
FIGS. 18-19, a general scheduling controller, as depicted in 
FIGS. 20-22, and a general scheduling controller with 
time-stamp, as depicted in FIGS. 23-24. 
A Double-buffer Operation 65 

The double-buffer scheduling and congestion controller, 
in FIGS. 18-19, is constructed of a central processing unit 
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(CPU), a random access memory (RAM) for storing the data 
packet, read only memory (ROM) for storing the controller 
processing program, and it operates in the following manner. 

In FIG. 18, data packets that arrive from the switching 
fabric 50 via link 51 in which their type or priority bits P1/P2 
are either 00 or 01 (35P in FIG. 14), i.e., CBR or VBR traffic, 
will be switched through the packet DMUX (demultiplexer) 
5 IS during odd ticks of the common time reference to buffer 
Ba via link 51-1, and during even ticks of the common time 
reference to buffer Bb via link 51-2. The data packets in 
which their P1/P2 bits are 00 are stored in the CBR part of 
the Ba and Bb buffers, and data packets in which their P1/P2 
bits are 01 are stored in the VBR part of the Ba and Bb 
buffers. 

In FIG. 18, data packet in which their type or priority bits 
P1/P2 are 10 will be switched through the packet DMUX 
(demultiplexer) 5 IS to the "best effort" buffer B-E via link 
51-be. This operation is specified in 45-02 in FIG. 19. The 
transmit buffer selection operation is controlled by the select 
signal 46A, which connects the double-buffer scheduling 
controller with the packet DMUX (demultiplexer) 51 S. 

Data packets are forwarded to the serial transmitter 49 
through the packet MUX (multiplexer) 47S and link 47C in 
FIG. 18. During the odd ticks of the common time reference 
packets are forwarded from buffer Bb via link 46-2, and 
during even ticks of the common time reference packets are 
forwarded from buffer Ba via link 46-1. During each of the 
time frame, first CBR packets (P1/P2 are 00) are forwarded 
and if there are no more CBR packets, then VBR packets 
(P1/P2) are forwarded. If during odd ticks of the common 
time reference buffer Bb is empty, data packet from the "best 
effort" buffer B-E are forwarded to the serial transmitter, and 
if during even ticks of the common time reference buffer Ba 
is empty, data packet from the "best effort" buffer B-E are 
forwarded to the serial transmitter. This operation is speci- 
fied in 45-02 in FIG, 19. The transmit buffer selection 
operation is controlled by the select signal 46B, which 
connects the double-buffer scheduling and congestion con- 
troller with the packet MUX (multiplexer) 47S. 
General Scheduling Controller 45A 

A more general scheduling controller 45 operation is 
described in FIGS. 20-22, which includes a scheduling and 
rescheduling controller 45A, a transmit buffer 45 C, and a 
select buffer and congestion controller 45D, as shown in 
FIG. 20. The scheduling and rescheduling controller 45A 
together with the select buffer controller 45D perform the 
mapping of the data packet into the time frame. The mapping 
is done on the scheduling and rescheduling controller using 
the PID 35C and the data packet time of arrival (TOA) 35T 
in order to determine the respective time frame in which the 
respective packet should be forwarded out of the output port. 
The details are presented below. Both controllers, 45A and 
45D, are constructed of a central processing unit (CPU), a 
random access memory (RAM) for storing the data, and read 
only memory (ROM) for storing the controller processing 
program. 

In the more general configuration, depicted in FIG. 20, 
data packets that arrive from the switching fabric 50 via link 
51 in which their priority bits 35P (P1/P2) are either 00 or 
01 (i.e., reserved CBR traffic, or VBR traffic) will be 
switched by controller 45A to one of the k transmit buffers 
in 45C: B-l, B-2, . . . , B-k. Each of the k buffers is 
designated to store packet that will be forwarded in each of 
the k time frames in every time cycle, that where defined in 
FIGS. 5 and 6. Another possible operation is to map the 
incoming packets separately to each of the time frames of a 
super-cycle. When a super-cycle mapping is implemented 



06/04/2004, EAST Version: 1.4.1 



US 6,272 ; 

17 

there are k*l transmit buffers in 45C: B-l, B-2, . . . , B-k*l, 
i.e., k buffers to each of the 1 cycles of a super-cycle. 

The actual program executed by the scheduling and 
rescheduling controller is described in FIG. 21. When a data 
packet is received from either the fabric via link 51 or from 5 
the select buffer and congestion controller 45D via link 45R, 
as specified in 45-03, the 35C, 35T and 35P in the data 
packet header are used to look-up the forward parameter 45F 
in the forwarding table 45B, as specified in 45-04. Next, the 
index i of the transmit buffer, between B-l and B-k, is 10 
computed in 45-05 by subtracting the time of arrival TOA 
35T from the common time reference CTR 002 and by 
adding the forward parameter 45F, and then switching the 
incoming data packet to transmit buffer B-i, as specified in 
45-05. 15 

Incoming data packets in which their priority bits 35P, . 
P1/P2, are either 10 (i.e., no n- reserved traffic or "best 
effort") or 11 (i.e., rescheduled packet) are switched by the 
scheduling and rescheduling controller to the transmit "best 
effort" buffer B-E via link 45-be. 20 

FIG. 22 depicts the operation of the select buffer and 
congestion controller 45D operation, which is responsive to 
the common time reference (CTR) tick 002. When CTR 
signal is received 45-11 the following operations are 
executed by controller 45D in 45-15: 25 

1. Send time frame delimiter (TFD) control signal 47 A to the 
serial transmitter 49; 

2. Forward back to controller 45A all unsent packets in 
transmit buffer B-f in which the 35P field in their header 

is not 11, i.e., it is not a rescheduled packet and set the 35P 30 
field to 11, i.e., a rescheduled packet (note that a packet 
can be rescheduled only once); If the number of data 
packets need to be rescheduled exceeds some predefined 
number, say n, then select at random n data packets and 
sends them back to the packet scheduling and reschedul- 35 
ing controller 45A via link 45R and discard the remainder 
of the packets. 

3. Increment the transmit buffer index f (i.e.,f;=f+l mod k, 
where k is the time cycle size in time frames). If the CBR 
part of buffer B-f is not empty 45-12, then it will send a 40 
data packet from transmit buffer B-f first CBR packets and 
then VBR packets, as specified in 45-16 and 45-13, else 

it will send a "best effort" data packet from the "best 

effort" buffer B-E as specified in 45-14. 
General Scheduling Controller 45A with Time-stamp 35TS 45 

FIGS. 23-24 describe a more general operation of the 45A 
controller since it incorporates into its scheduling and 
rescheduling operation the time stamp value in the data 
packet header 35TS. The time-stamp 351S can be generated, 
for example, by an Internet real-time protocol (RTP) in 50 
which its data packet header format is depicted in FIG. 23. 
Alternatively the time-stamp is generated by a predefined 
one of the switches in the system, or alternatively the time 
stamp is generated at the respective end station for inclusion 
in the respective originated data packet header. 55 

The actual program executed by the 45A controller when 
the data packet header include a time-stamp 35TS field (see 
FIGS. 14 and 23) is described in FIG. 24. When a data 
packet is received from either the fabric via link 51 or from 
the select buffer and congestion controller 45D via link 45R, 60 
as specified in 45-21, the time-stamp 35TS, 35C, 35T and 
35Pin the data packet header are used to look-up the forward 
parameter 4SF in the forwarding table 45B, as specified in 
45-22. Next, the index i of the transmit buffer, between B-l 
and B-k, is computed in 45-23 by subtracting the time of 65 
arrival TOA35T from the common time reference CTR 002 
and by adding the forward parameter 45F, and then switch- 
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ing the incoming data packet to transmit buffer B-i, as 
specified in 45-23. Incoming data packets in which their 
priority bits 35P, P1/P2, are either 10 (i.e., non-reserved 
traffic or "best effort") or 11 (i.e., rescheduled packet) are 
switched by the scheduling and rescheduling controller to 
the transmit "best effort" buffer B-E via link 45-be. 

In another configuration of the scheduling and resched- 
uling controller 45A, the time stamp 35TS is used in order 
to determine a delay count. Such a delay count can be 
realized by the time stamp 35TS value from the current 
common time reference 002 value. When the delay count is 
either greater than a first predefined value or smaller than a 
second predefined value the data packet is discarded The 
first and second predefined values can be determined by 
looking them up with the PID 35 C in the forwarding table 
45B. 

From the foregoing, it will be observed that numerous 
variations and modifications may be effected without depart- 
ing from the spirit and scope of the invention. It is to be 
understood that no limitation with respect to the specific 
apparatus illustrated herein is intended or should be inferred. 
It is, of course, intended to cover by the appended claims all 
such modifications as fall within the scope of the claims. 

What is claimed is: 

1. A system for transferring packet data from at least one 
source to at least one destination wherein the transfer of the 
data packet is provided during respective ones of a plurality 
of predefined time intervals, wherein each of the predefined 
time intervals is comprised of a plurality of predefined time 
frames, comprising: 

a virtual pipe comprising at least two switches intercon- 
nected via communication links, each of the switches 
having a plurality of input ports and output ports, each 
with a unique address for receiving the data packets; 

a routing controller for mapping each of the data packets 
that arrives at each one of the input ports to a respective 
one of the output ports; 

a common time reference signal coupled to each of the 
switches; 

a packet scheduling and rescheduling controller, wherein 
for each switch there is a first scheduled time within a 
first predefined time frame within which a respective 
packet is scheduled to be transferred out of the respec- 
tive switch determined by said packet scheduling and 
rescheduling controller; 

wherein the time frames and the first scheduled time 
frame are determined responsive to the common time 
reference; and 

wherein the position of a data packet in a time fame is 
arbitrary. 

2. The system as in claim 1, wherein for each of the time 
frames there is a defined bandwidth limiting the capacity to 
transmit the packets during the time frame, the system 
further comprised of: 

a select buffer and congestion controller for determining 
congestion responsive to the defined bandwidth being 
exceeded by the scheduling controller and for a respec- 
tive one of the time frames; 

a rescheduling controller responsive to the determination 
of congestion for a particular one of the time frames, 
for rescheduling selected ones of the packets to be 
associated with a second predefined time within a 
second predefined time frame for transfer of the respec- 
tive packets out from the respective switch. 

3. The system as in claim 1, wherein there are a plurality 
of virtual pipes, wherein there is an overlap of at least one 
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of the switches within at least two of the virtual pipes, 20. The system as in claim 1, wherein the common time 

wherein the same link between two adjacent switches is used reference signal coupled to each of the switches conforms to 

simultaneously by the at least two of the virtual pipes. UTC (Coordinated Universal Time) standard. 

4. The system as in claim 1, wherein there are a plurality 21. The system as in claim 20, wherein the super cycle 
of the virtual pipes, wherein the data packets for at least two 5 duration is equal to one second as measured using the UTC 
of the virtual pipes are transferred during the same pre- (Coordinated Universal Time) standard. 

defined time frame. 22. The system as in claim 20, wherein the super cycle 

5. The system as in claim 2 wherein the rescheduling of duration is equal to a predefined number of seconds as 
selected ones of the packets are determined at random from measured using the UTC (Coordinated Universal Time) 
a predefined set of the time frames. 10 standard. 

6. The system as in claim 2 wherein each of the data J 23 ^ s y stei * as in claiin 20, wherein the super cycle 
packets is comprised of priority bits, wherein the resched- duration is equal to a predefined fraction of a second 
uling of selected ones of the packets is responsive to the mea ^ d usm S the UTC (Coordinated Universal Time) 
priority bits of the respective one of the data packets. a ' ... 

7. The system as in claim 1, wherein there are a plurality 1S 24 ™<: system as in clatm 18, wherein the time fames 
of virtual pipes, wherein each of the data packets is associ- <^°«ated ™* particular ones of the switches are 
ated with one of the respective virtual pipes, wherein for re P^ Utlve w,bm each of me dme c y cles W1,hm su P er 
each virtual pipe there is an associated predefined allotted c ^?5 ' , . „ , 

packet capacity for each time frame, the system further 25 ' ™ e s y stem 35 m claun comprising: 

comprising - 20 a DU ^ er aR d congestion controller for mapping a 

logic responsive to the first scheduled time and to the ™ respective one of the time frames for output from a first 

allotted data packet capacity for each of the respective one ° f °* s ™ tches t0 asec ° nd ^specUve one of the 

time frames for each of the respective virtual pipes for bme frames G» output ™the communications lmk to 

marking respective ones of the packets for reschedul- cond one of . the f Wlt <*>f " L . , _ , , 

• 26. The system as in claim 25, wherein each of the data 

8. Tte system as in claim 7, wherein the marked respec- * P ackets * encoded as a stream of dala i 

tive packet is rescheduled to the next sequential time frame wherein a time frame delimiter is inserted into the stream 

of the associated virtual pipe, responsive to the marking of of data responsive to the scheduling controller and 

the packets for rescheduling. transmit buffer. 

9. The system as in claim 7, wherein each of the packets 30 27 • ^ svstem as in claim X > wherein ^ere are a plurality 
is comprised of a header with a time delay count field, of related local clock signals derived from the common time 

wherein the marked respective packet is rescheduled for re crcDCC S1 £° * 

a subsequent one of the time frames responsive to the wherein a synchronization envelope is associated with the 

delay count field being less than a predefined threshold. . common time reference, 

10. The system as in claim 9, wherein the delay count field 35 wherein two adjacent synchronization envelopes of two 
is a time stamp and said delay count is determined by adjacent time frames are non-overlapping; and 
computing the time difference between the time stamp value wherein all the related local clock signals fall within a 
and the common time reference value. respective one of the synchronization envelope. 

11 . The system as in claim 9, wherein the marked respec- 28 . The system as in claim 27, wherein between every two 
tive data packet is discarded from scheduling for the respec- 40 successive time frames the select buffer controller sends a 
tive virtual pipe, responsive to the delay count being greater time frame delimiter (TFD) signal via the serial transmitter, 
than the predefined threshold. 29. The system as in claim 28, wherein the time frame 

12. The system as in claim U, wherein the respective delimiter can be sent as an encoded control codeword within 
discarded data packet from said virtual pipe is forwarded the data packet without adversely affecting the transmission 
independent and outside of the respective virtual pipe. 45 of said data packet. 

13. The system as in claim U, wherein the respective 30. The system as in claim 29, wherein all data packets 
discarded data packet from said virtual pipe is forwarded as received between two successive time frame delimiters 
"best effort" data packets. (TFDs), received by the routing controller, are associated 

14. The system as in claim 1, wherein the predefined time with a unique one of the time frames. 

interval is comprised of a fixed number of contiguous time 50 31. The system as in claim 1, wherein the communication 

frames comprising a time cycle. links are wireless communication links. 

15. The system as in claim 14, wherein the time cycles are 32. The system as in claim 1, wherein the communication 
contiguous. links are wireless communication links between at least one 

16. The system as in claim 14, wherein the time frames, of a ground station and a satellite and between two satellites 
associated with a particular one of the switches within the 55 orbiting the earth. 

virtual pipe, are associated for the same respective switches 33. The system as in claim 1, wherein the data packets are 

in all the time cycles. at least one of Internet protocol (IP) data packets, asynchro- 

17. The system as in claim 16, wherein the time frames nous transfer mode (ATM) cells. 

associated with the particular ones of the switches are 34. The system as in claim 1, wherein the data packets 

associated with one of input into and output from the 60 forwarded over the same virtual pipe each have an associ- 

respective switches. ated pipe identification (PID); 

18. The system as in claim 15, wherein a fixed number of wherein the routing controller is responsive to the asso- 
a plurality of contiguous ones of the time cycles comprise a ciated PID. 

super cycle; wherein the super cycle is periodic. 35. The system as in claim 34, wherein the PID is at least 

19. The system as in claim 1, wherein the common time 65 one of an Internet protocol (IP) address, an asynchronous 
reference signal coupled to each of the switches comes from transfer mode (ATM) a virtual circuit identifier, and a virtual 
GPS (Global Positioning System). path identifier (VCI/VPI). 
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36. The system as in claim 1, wherein the routing con- 
troller utilizes at least one of Internet protocol version 4 
(IPv4) and Internet protocol version 6 (IPv6) addresses, 
Internet MPLS (multi protocol label swapping or tag 
switching) labels, ATM virtual circuit identifier and virtual s 
path identifier (VCI/VPI), and IEEE 802 MAC (media 
access control) addresses for mapping from said input port 

to said output port. 

37. The system as in claim 1, further comprising a 
scheduling controller, wherein each of the data packets is 
comprised of a header, having an associated time stamp, 
wherein for each of the mappings by the routing controller, 
there is an associated mapping by the scheduling controller, 
wherein the scheduling controller provides for mapping of 
each of the data packets responsive to the respective asso- 
ciated time stamp and an associated time out, wherein the 15 
time out is associated with a unique one of the predefined 
time frames. 

38. The system as in claim 37, wherein the time-stamp is 
generated by an Internet real-time protocol (RTP). 

39. The system as in claim 37, wherein the time-stamp is 20 
generated by a predefined one of the switches. 

40. The system as in claim 37, wherein each of the data 
packets is originated from an end station, wherein the 
time -stamp is generated at the respective end station for 
inclusion in the respective originated data packet. 2 5 

41. A system for switching data packets, the data packets 
comprising a header having a PID field, from at least one 
source to at least one destination, wherein the transfer of the 
data packets is provided during respective ones of a plurality 

of predefined time intervals, wherein each of the predefined 3Q 
time intervals is comprised of a plurality of predefined time 
frames, said system comprising: 

a plurality of switches with a plurality of uniquely addres- 
sable input ports and output ports; 
a common time reference signal coupled to each of the 35 
switches; 

wherein the common time reference signal is parti- 
tioned into time frames; 
wherein a predefined number of contiguous time 

frames are grouped into a time cycle; 40 
wherein a predefined number of contiguous time cycles 
are grouped into a super cycle; 
a routing controller, coupled to the input ports for deter- 
mining which of the plurality of output ports said data 
packet will be forwarded to, and for attaching a time of 45 
arrival (TOA) to the respective data packet; 
a packet scheduling and rescheduling controller for 
assigning selected predefined time frames for transfer 
into and out from each of the respective switches 
responsive to the time of arrival, the unique address of 50 
the respective input port, and the PID field in the data 
packet header; and 

wherein for each switch there is a first predefined time 
frame within which a respective data packet is trans- 
ferred into the respective switch, and a second pre- 55 
defined time frame within which the respective data 
packet is forwarded out of the respective switch. 

42. The system as in claim 41, wherein the position of said 
data packet in said second predefined time frame is arbitrary. 

43. The system as in claim 42, wherein the time of arrival 60 
reflects the UTC time and is represented as a time frame 
number within a time cycle and as time cycle number within 

a super cycle. 

44. The system as in claim 43, wherein the second 
predefined time frame within which the respective data 65 
packet is forwarded out of the respective switch is deter- 
mined responsive to UTC and PID. 
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45. The system as in claim 41, wherein a select buffer and 
congestion controller determines that there is no capacity for 
a data packet in said second predefined time frame out of 
said switch; 

wherein the packet scheduling and rescheduling control- 
ler reschedules said data packet for output in another 
time frame out of said switch. 

46. The system as in claim 41, wherein for each switch 
there is a predefined time difference, measured in time 
frames, between the first predefined time frame within 
which the respective data packet is transferred into the input 
port of respective switch and the second predefined time 
frame within which the respective data packet is forwarded 
out of the output port of respective switch. 

47. The system as in claim 46, wherein the position of said 
data packet in said second predefined time frame is arbitrary. 

48. The system as in claim 46, wherein for each switch the 
predefined time difference is predefined for each of the PID 
in the data packet header. 

49. The system as in claim 46, wherein for each switch the 
predefined time difference is predefined for each of the time 
frames within a time cycle, for each of the PIDs in the data 
packet header. 

50. The system as in claim 46, wherein for each switch the 
predefined time difference is predefined for. each of theme 
frames within the time cycle, for all the time cycles within 
the super cycle, for each of the PIDs in the data packet 
header. 

51. The system as in claim 41, wherein there are no 
scheduled data packet to be transmitted in a time frame, 
"best effort" data packets are transmitted. 

52. The system as in claim 45, wherein the data packets 
are scheduled for transmission by the scheduling and 
rescheduling controller, wherein responsive to the select 
buffer and congestion controller determining that there are 
no scheduled data packets to be transmitted in a given time 
frame, the select buffer and congestion controller forwards 
"best effort" data packets for output. 

53. A system for scheduling data packets on a switch 
comprised of a plurality of uniquely addressable input ports 
and a plurality of uniquely addressable output ports, wherein 
the transfer of the data packets is provided during respective 
ones of a plurality of predefined time intervals, wherein each 
of the predefined time intervals is comprised of a plurality of 
predefined time frames, said system comprising: 

a common time reference signal coupled to each of the 
switches; 

wherein the common time reference partitioned into 
time frames; 

wherein a predefined number of contiguous k time 
frames are grouped into a time cycle; wherein k is at 
least 1; 

wherein a predefined number of contiguous 1 time 
cycles are grouped into a super cycle; wherein 1 is at 
least 1; 

a routing controller with a routing table for selecting at 
least one of the output ports said data packets will be 
forwarded to; 

wherein the routing controller is coupled to the input port 
and provides for attaching a time of arrival (TOA) to 
incoming ones of the data packets; 

wherein the time of arrival is derived from the common 
time reference and is represented as a time frame 
number within a time cycle and as a time cycle number 
within a super cycle; 

a switching fabric for coupling of data packets between 
the input and the output ports; 
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a packet scheduling and rescheduling controller for sched- 
uling respective ones of the data packets for transfer 
during predefined associated ones of the time frames. 

54. The system as in claim 53, further comprising a select 
buffer and congestion controller; 5 

wherein the packet scheduling and rescheduling control- 
ler is responsive to the reception of a data packet from 
the switching fabric for assigning a first predefined time 
frame for transferring said data packet out from the said 
switch responsive to the time of arrival attached to the 10 
data packet by the routing controller, the unique 
address of input port, and the PID in the data packet 
header; 

wherein the select buffer and congestion controller deter- 
mines if capacity is available in said first predefined 15 
time frame for transferring said data packet out from 
the said switch; and 

wherein the packet scheduling and rescheduling control- 
ler is responsive to the reception of the data packet from ^ 
the select buffer and congestion controller for assigning 
a second predefined time frame for transferring said 
data packet out from the said switch when capacity is 
not available. 

55. The system as in claim 50, further comprising a ^ 
random access memory partitioned into plurality of buffers; 

wherein the random access memory is partitioned into 
three buffers; 

wherein the first buffer stores data packets to be forwarded 
from said switch during odd time frames as measured 30 
by the common time reference; 

wherein the second buffer stores data packets to be 
forwarded from said switch during even time frames as 
measured by the common time reference; 

wherein the third buffer stores "best effort" data packets 35 
to be forwarded from said switch during odd and even 
time frames, whenever the first and second buffers are 
empty. 

56. The system as in claim 55, wherein said first buffer 
and said second buffer are further partitioned into two parts 40 
(i) constant bit rate (CBR) and (ii) variable bit rate (VBR). 

57. The system as in claim 56, wherein during each time 
frame the select buffer and congestion controller first trans- 
fers data packets from the CBR part and then from the VBR 
part of said buffer. 45 

58. The system as in claim 53, further comprising random 
access memory, 

wherein the random access memory is partitioned into 
k+1 buffers; 

wherein first k buffers provide for storing data packets to 50 

be forwarded from said switch during one of the k time 

frames in each of said time cycle; 
wherein the k+1 buffer provides for storing "best effort" 

data packets; and 55 
wherein said first k buffers are further partitioned into two 

parts (i) constant bit rate (CBR) and (ii) variable bit rate 

(VBR). 

59. The system as in claim 58, further comprising a select 
buffer and congestion controller, 60 

wherein in each of the k time frames of the time cycle the 
select buffer and congestion controller forwards data 
packets from the corresponding one of the k buffers; 

wherein said select buffer and congestion controller for- 
wards data packets first from the CBR part of said 65 
buffer and then from the VBR part of said buffer and 
when said VBR part is empty said select buffer and 



congestion controller forwards data packets out of said 
"best effort" buffer. 

60. The system as in claim 53, further comprising a select 
buffer and congestion controller and a random access 
memory, 

wherein the random access memory is partitioned into (1 

times k)+l buffers; 
wherein first (1 times k) buffers provide for storing data 

packets to be forwarded from said switch during one of 

the (1 times k) time frames in each of said super cycle; 
wherein the (I times k)+l buffer provides for storing "best 

effort" data packets; and 
wherein said first (1 times k) buffers are further partitioned 

into two parts (i) constant bit rate (CBR) and (ii) 

variable bit rate (VBR). 

61. The system as in claim 60, wherein in each of the (I 
times k) time frames of the super cycle, the select buffer and 
congestion controller forwards the respective data packets 
from the corresponding one of the (1 times k) buffers; 

wherein said select buffer and congestion controller for- 
wards the respective data packets first from the CBR 
part of said buffer, and then from the VBR part of said 
buffer, and wherein the select buffer and congestion 
controller forwards data packets out of said "best 
effort" buffer when the said VBR part is empty. 

62. The system as in claim 53, farther comprising a select 
buffer and congestion controller, wherein between every two 
successive time frames the select buffer and congestion 
controller sends a time frame delimiter (TFD) signal via the 
serial transmitter. 

63. The system as in claim 62, wherein the time frame 
delimiter can be sent as an encoded control codeword in the 
respective data packet without adversely affecting the trans- 
mission of said data packet. 

64. A system for switching data packets comprised of a 
data header having a PID field and a time stamp field, from 
at least one source to at least one destination, wherein the 
transfer of the data packets is provided during respective 
ones of a plurality of predefined time intervals, wherein each 
of the predefined time intervals is comprised of a plurality of 
predefined time frames, said system comprising: 

a plurality of switches with a plurality of uniquely addres- 
sable input and output ports; 

a common time reference signal coupled to each of the 
switches; 

wherein the common time reference is partitioned into 
time frames; 

wherein a predefined number of contiguous time 

frames are grouped into a time cycle; 
wherein a predefined number of contiguous time cycles 
are grouped into a super cycle; 

a routing controller at the input port for determining 
uniquely the output port for forwarding of a respective 
data packet responsive to the PID field in the respective 
data packet header; 

a scheduling and rescheduling controller with a forward- 
ing table for assigning predefined time frame for trans- 
fer out from each of the respective output ports respon- 
sive to the time stamp field, the unique address of the 
input port, and the PID field in the data packet header. 

65. The system as in claim 64, wherein the predefined 
time frame for transferring the data packet out is determined 
by adding a predefined number of time frames to the time 
stamp field in the data packet header. 

66. The system as in claim 65, wherein the number of 
predefined time frames added to the time stamp field in order 
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to determine transferring time frame of said data packet out, 
is determined responsive to looking this number up in the 
forwarding table in the scheduling controller using the P1D 
in the data packet header as an index to said forwarding 
table, 

67. The system as in claim 64, wherein the scheduling and 
rescheduling controller computes the time difference in time 
frames between common time reference and the time stamp 
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in the data packet header and for discarding the respective 
data packet when said time difference is above a predefined 
time threshold. 

68. The system as in claim 67, wherein the time threshold 
value is determined by using the PID to look-up said value 
in the forwarding table. 
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